Skip to content

fix(flows): inherit unset streaming fields on after_model_callback replacement - #7036

Closed
Bruce-Yii wants to merge 2 commits into
google:mainfrom
Bruce-Yii:fix/7035-inherit-streaming-fields-after-model-callback
Closed

Bruce-Yii wants to merge 2 commits into
google:mainfrom
Bruce-Yii:fix/7035-inherit-streaming-fields-after-model-callback

Conversation

@Bruce-Yii

@Bruce-Yii Bruce-Yii commented Sep 7, 2026 •

Copy link
Copy Markdown
Contributor

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

Problem:

after_model_callback may return a replacement LlmResponse ("the actual model response will be ignored and the provided content will be returned to user"). During streaming, the callback runs on every response including partial=True deltas, and the replacement was used wholesale. Since LlmResponse.partial defaults to None — and unset means final per the streaming protocol — any callback that rebuilds the response silently converted every delta into a final response. Runner persists every non-partial event, so each delta fragment was appended to the session as a separate complete model event (4 persisted events instead of 1 in the issue's repro), and SSE clients saw N "final" responses. The same field-loss class affects turn_complete (live path closes its request queue on it).

Solution:

In handle_after_model_callback in src/google/adk/flows/llm_flows/_model_response_finalizer.py (the single shared choke point for plugin + agent callbacks, covering both the live and SSE call sites), inherit partial/turn_complete from the replaced response when the replacement leaves them unset. An explicitly set value is always respected. Inheritance returns a model_copy only when a field actually needs filling, so explicitly complete replacements keep their identity and callback-owned objects are never mutated. usage_metadata is deliberately left out of this change to keep the fix minimal (noted as residual risk below).

Note: this branch was rebased onto current main (2284c58) via merge (no force). Upstream has since refactored the callback handling out of base_llm_flow.py into _model_response_finalizer.py; the fix follows it to the new choke point with identical semantics.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

New tests in tests/unittests/flows/llm_flows/test_base_llm_flow_partial_handling.py (5 added, file now 9 tests):

  • test_after_model_callback_replacement_inherits_partial — rebuilt replacement keeps the delta's partial=True; the callback's object is not mutated.
  • test_after_model_callback_replacement_inherits_turn_complete.
  • test_after_model_callback_replacement_explicit_partial_respected — explicit partial=False wins (guards against over-correction).
  • test_after_model_callback_replacement_without_streaming_keeps_identity — non-streaming pass-through untouched.
  • test_run_async_sse_rebuilt_responses_stay_partial — flow-level SSE run: deltas stay partial, final stays final.

Fail-before verified on pristine current main (2284c58): the issue's repro shows CONTROL [True, True, True, None] with 1 persisted event versus BUG [None, None, None, None] with 4 persisted events.

pytest results (Python 3.12, Windows), on the merged branch:

  • tests/unittests/flows/llm_flows/test_base_llm_flow_partial_handling.py: 9 passed
  • test_base_llm_flow.py + test_model_response_finalizer.py + test_live_model_callbacks.py + test_model_callbacks.py: 155 passed
  • tests/unittests/flows/llm_flows/ (minus 2 files with pre-existing collection errors for uninstalled optional openai/speech deps, also present without this change): 705 passed, 1 skipped, plus 1 pre-existing failure caused by the uninstalled optional anthropic package (fails identically on pristine main)
  • test_llm_agent_callbacks.py + callback tests + tests/unittests/runners: 103 passed, 1 skipped, 3 xfailed
  • ruff check clean; pyink and isort clean. tox was not run (environment limitation).

Manual End-to-End (E2E) Tests:

Ran the issue's verbatim repro script (InMemoryRunner, fake streaming model, no API key) on pristine current main and on this branch:

  • Before (pristine 2284c58): BUG run events all partial=None/final=True, 4 persisted model events.
  • After (this branch at 028fd0a): events partial=True/True/True/None, 1 persisted model event (Hello [REDACTED] world.), matching the issue's documented model_copy workaround shape.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules. (N/A — no dependents.)

Additional context

  • Residual risk: usage_metadata has the same loss shape (a rebuilt response drops token accounting). Left out deliberately for minimal scope; happy to follow up if maintainers want it covered too.
  • Note: the Google CLA is signed (cla/google check is green).
  • AI assistance: an AI coding agent performed the reproduction, implementation, and tests under my direction; I reviewed the diff and take responsibility for it.

@google-cla

google-cla Bot commented Sep 7, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

# Conflicts:
#	src/google/adk/flows/llm_flows/base_llm_flow.py
copybara-service Bot pushed a commit that referenced this pull request Sep 10, 2026
…es a response

Merge #7036

Fixes #7035

Co-authored-by: George Weale <gweale@google.com>
PiperOrigin-RevId: 979343383
@adk-bot

adk-bot commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Thank you @Bruce-Yii for your contribution! 🎉

Your changes have been successfully imported and merged via Copybara in commit 39e4538.

Closing this PR as the changes are now in the main branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merged [Status] This PR is merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

after_model_callback: a replacement LlmResponse silently loses partial during SSE streaming - every delta is persisted to the session as a final event

3 participants